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ABSTRACT 

The intelligent visual system detects anomalies or defects in real time under normal lighting 
operating conditions. The application is basically a learning machine that integrates fuzzy logic 
(FL), artificial neural network (ANN), and generic algorithm (GA) schemes to process the image, 
run the learning process, and finally detect the anomalies or defects. The system acquires the image, 
performs segmentation to separate the object being tested from the background, preprocesses the 
image using fuzzy reasoning, performs the final segmentation using fuzzy reasoning techniques to 
retrieve regions with potential anomalies or defects, and finally retrieves them using a learning 
model built via ANN and GA techniques. 

FL provides a powerful framework for knowledge representation and overcomes uncertainty and 
vagueness typically found in image analysis. ANN provides learning capabilities, and GA leads to 
robust learning results. An application prototype currently runs on a regular PC under Windows NT, 
and preliminary work has been performed to build an embedded version with multiple image 
processors. 

The application prototype is being tested at the Kennedy Space Center (KSC), Florida, to visually 
detect anomalies along slide basket cables utilized by the astronauts to evacuate the NASA Shuttle 
launch pad in an emergency. The potential applications of this anomaly detection system in an open 
environment are quite wide. Another current, potentially viable application at NASA is in detecting 
anomalies of the NASA Space Shuttle Orbiter's radiator panels. 
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INTRODUCTION 

Automated real-time visual inspection under normal lighting is a very complex task. The main 
problems (which include image vagueness due to lack of proper lighting, extremely high data flow, 
noise influence, interclass similarity and interclass diversity, large number of defect classes, dynamic 
defect population, and economical justification) are not solved yet. 

Handling uncertainty on image features (image enhancement, edginess grading, background 
extraction, noise elimination, etc.) and performing visual classification in real time are the main 
problem characteristics of a visual anomaly detection system. Computer vision, in general, and 
particularly anomaly detection under normal lighting conditions are very difficult problems. This is 
in part because of the many levels of uncertainty inserted into the data by a variety of processes from 
the original sensor through several levels of interpretation. 

Mainstream approaches to dealing with this uncertainty include the use of thresholds, which are 
difficult to choose and cause information loss, and probabilistic methods, which rely on knowledge 
about prior distribution that is often not available. Other approaches are based on a morphological 
filtering method and neural-network classifier, but they face problems performing image threshold 
and feature extraction for some specific anomalies or defects under normal lighting conditions. 
Integrating the three constituents of soft computing - FL, AN, and GA - solves these problems. 

Image analysis leading to either detection of anomalies or performance of pattern recognition is not 
only an "inverse inspection problem" without a single mathematical solution, but is also complicated 
by external sources of uncertainty such as the conditions of the image formation and the final 
decision-making approach. Thus, the need for dealing with uncertainty in computer vision is well 
accepted. However, most of the work in this area has used fixed thresholds or probabilities 
approaches. Although fuzzy systems are basically out of the mainstream of computer vision, they 
pose great value for addressing uncertainty issues that are not adequately dealt with by current 
methods. 

SYSTEM DESCRIPTION 

The system acquires the images from a video camera via either an analog or digital image grabber. 
Using fuzzy reasoning, the image undergoes preprocessing that includes primary segmentation, 
object extraction, enhancement, filtering, and edginess detection. An adaptive fuzzy threshold 
binaries the preprocessed image, and a selective blob detector picks image regions with high 
edginess activity. The system computes morphological, geometrical, and fuzzy edginess grading 
values for each region and uses them as inputs of the learning feature via GA-ANN schemes. A 
schematic of the system is shown in figure 1 . ; 

The system has several innovative features to perform image analysis based on fuzzy reasoning and 
integrates them with a learning machine built using ANN and GA schemes. The image analysis 
needed to perform enhancement, filtering, edginess grading, and segmentation is based on a unique 
and innovative fuzzy reasoning approach. The application is basically a general-purpose and real- 
time anomaly detection system that does not require special lighting and can be embedded to a 
portable application in an open environment. 
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Figure 1. Schematic of the intelligent visual anomaly detection system. 

An innovative concept of edginess grading based on fuzzy reasoning implemented here makes it 
possible to identify in real time potential anomaly spots on the images. This approach quantifies the 
degree of edginess and uses its value as an input for the learning scheme via ANN and GA. The 
learning scheme also computes several morphological and geometrical features of these spots and 
uses them as inputs. 

Visual analysis, in general, requires a large amount of CPU power leading to the degradation of its 
real-time performance. Anomaly detection systems have to be run under a “hard” real-time 
constraint. The code of all image operators based on fuzzy reasoning and used to implement the 
image preprocessing stages (segmentation, enhancement, filtering, edginess grading, and 
binarization) is optimized to minimize execution overhead and keep the system running under 
“hard” real-time performance requirements. 

The system may run on either a regular PC under Windows NT or an embedded version for 
portability. Hardware required to build the system includes a video camera, an image grabber, an 
image processor (a PC processor or a stand-alone image processor in the embedded application), and 
the respective peripheral devices (monitor, mouse, keyboard, and data storage medium) attached to 
the image processor. The software development tools used in the implementation of this system 
include Microsoft Visual C++, Microsoft Foundation Class (MFC), and two. commercial software 
packages: Wit image analysis software from Logical Vision and NeuroShell Classifier from Ward 
Systems. 

Wit software allows the developer to build customized image operators in C and a front end to make 
the application friendly and transparent to the end user. We designed and built most of the image 
operators, including those needed to perform image preprocessing, via fuzzy reasoning. The ANN 
scheme is based on Probabilistic Neural Nets (PNN) but is formulated a little differently, based on 
GA as opposed to a network structure. 
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PNN’s instantly train, unlike back propagation. In fact, they do not really train at all; they 
memorize, and the only freedom they have to generalize is to vary the smoothing factors. So the 
approach used in NeuroShell memorizes all of the patterns in the training set. Then it begins 
generating genetic individuals with GA. Each individual is a set of smoothing factors, which, when 
applied to PNN, makes a complete net. That net is tested against the test set, and the results become 
the GA fitness function. The GA generates the smoothing factors that perform the best against the 
test set, but all nets thus generated by the GA are trained on the training set (memorized). 
NeuroShell Classifier contains this new GA-PNN network paradigm. NeuroShell Classifier is 
spawned via DLL functions to perform training and firing of the GA-PNN classification net. 

On the end-user side, all the features involved in the system are completely transparent. The end 
software product offers a friendly and easy-to-use template (all built via MFC) with a menu to select 
and perform training, analysis, and detection. The system has two end products - the development 
software package that includes all the system features and the run-time package designed to just run 
the anomaly detection system in the field. 

FUZZY REASONING AND ANN INTEGRATION 

Visual inspection faces many levels of uncertainty inserted into the data by a variety of processes 
from the original sensor through several levels of interpretation. The integration of the three 
constituents of soft computing - FL, ANN, and GA - is used to manage these levels of uncertainty. 
Soft computing differs from conventional (hard) computing in that, unlike hard computing, it is 
tolerant of imprecision, uncertainty, and partial truth. In effect, the role model for soft computing is 
the human mind. 

Fuzzy reasoning and ANN lead to a symbiotic relationship in which fuzzy systems provide a 
powerful framework for knowledge representation, while ANN provides learning capabilities and 
exceptional suitability for computationally efficient hardware implementation. Generating optimal 
smoothing factors via GA approach significantly enhances the performance and robustness of the 
PNN. 

The anomaly detection system uses a fuzzy reasoning approach to perform image preprocessing, 
adaptive image binarization, and image edginess grading. The fuzzy reasoning leads to the 
identification of image regions with important edginess activity. An additional system module 
retrieves those regions; computes morphological, geometric, and fuzzy edginess grading values for 
each region; and uses them to perform training and generate the respective ANN model. 

PROTOTYPE TEST - 

NASA has built an inspection mechanism called Cable and Line Inspection Mechanism (CLIM) to 
provide a means of automated image acquisition of the fourteen 1000-foot slidewires used in the 
emergency egress system for the NASA Space Shuttle at KSC. CLIM eliminates the hazardous, 
manpower-intensive, and time-consuming methods previously required to manually maintain the 
emergency egress system at peak performance. The inspection of the emergency egress system 
using CLIM still requires having a person continuously watch cable images for more than 48 hours. 
The intelligent anomaly detection system will be integrated to CLIM to completely eliminate the 
need for human intervention in detecting anomalies; the system will not only detect the anomaly, but 
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also document it. Figure 2 shows a picture of CLIM climbing one of the fourteen 1000- foot 
slidewires used in the emergency egress system for the NASA Space Shuttle at KSC. 



Figure 2. CLIM being tested at one of the slidewires used in the emergency 
egress system for the Space Shuttle at the Kennedy Space Center. Images 
acquired by CLIM are being used to test the intelligent real-time anomaly 
detection system. 

Images acquired by CLIM are being used to test the intelligent real-time anomaly detection system. 
Figures 3.1 through 3.5 show a real-time sequence of image events performed by the detection 
system: image acquired by CLIM (figure 3.1), cable image extracted from the background and 
preprocessed via fuzzy reasoning (figure 3.2), image binanzation via adaptive fuzzy threshold 
(figure 3.3), identification of regions with potential anomalies (figure 3.4), and final anomaly 
detection (figure 3.5). 

The anomaly shown in figure 3.1 was one of four other similar anomaly occurrences rated under the 
same type or class to test the application. This anomaly classification identifies small strand pieces 
slightly separated from the cable main strands. All five anomalies were located in 40 feet of cable, 
and all five anomalies were repeatedly detected by the system. An average of 4 false anomalies were 
detected in all runs (40 feet each run). Considering the number of points used for training (only five 
for anomaly and more than 400 for normal), the system performance supersedes the original 
expectation. The true-true detection rate was 100 percent and the true-false rate was lower than 0.02 
percent. Further tests using different anomaly and normal classifications are currently being 
planned. 
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Figure 3.1. Image acquired by CLIM holding a mild anomaly. 
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Figure 3.3. Adaptive fuzzy image threshold and binary image generation. 



Figure 3.4. Image skeleton indicating perimeter of individual regions with potential anomalies. 
Region 0 contains an actual mild anomaly (see figure 3.1). 
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Figure 3.5. Mild anomaly detection in real time indicated by a rectangle corresponding to region 0. 


CONCLUSIONS 

The CLIM project, which has been designed to visually document the slide basket cables utilized by 
the astronauts to evacuate the NASA Shuttle launch pad in an emergency, is being used to test the 
system. Preliminary tests, using images recorded from a slide basket cable mounted in the 
laboratory, show promising results. 

The potential applications of this anomaly detection system in an open environment are quite wide 
because there is no need for special lighting. One immediate application at KSC is the detection of 
anomalies of the NASA Space Shuttle Orbiter's radiator panels. Potential applications include 
automated visual inspection on spacecraft, ground, and flight equipment. The embedded portable 
version of this system makes the visual inspection of this type of equipment (including spacecraft 
fuselage) quite feasible in an open environment. 

The fuzzy operators built to preprocess the image can be incorporated into other computer vision 
systems. Current visual systems that rely on clustering, regardless of the space in which clustering 
occurs, and the visual systems that are based on thresholds can be generalized to use fuzzy 
thresholding to delay commitment. Finally, probabilistic visual systems that rely on prior 
distribution that is not really available can be recast as systems that match input data to fuzzy 
models. 


8 



REFERENCES 


Caudill, M., and C. Butler, Naturally Intelligent Systems, The MIT Press, Cambridge, 

Massachusetts, 1992. 

Chen, C.H., Fuzzy Logic and Neural Network Handbook, McGraw-Hill (Series on Computer 
Engineering), New York, 1992. 

Chi, Z., H. Yan, and T. Pham, “Fuzzy Algorithms: With Applications to Images Processing and 

Pattern Recognition,” Advances in Fuzzy Systems - Applications and Theory, Vol. 10, World 
Scientific Publishing Co., Pte. Ltd., Singapore. 

Cox, E., Fuzzy Systems Handbook, 2d ed., Academic Press, AP Professional, 1998. 

Jain, A.K., Fundamentals of Image Processing, Prentice-Hall, Englewood Cliffs, New Jersey, 1998. 

Laplante, P.A., and A.D. Stoyenko, Real-Time Imaging: Theory, Techniques, and Applications, 
IEEE Press, 1996. 

Lie, W.N., “Automatic Target Segmentation by Locally Adaptive Image Thresholding,” IEEE 
Trans. Image Processing, Vol. 4, July 1995, pp. 1036-1041. 

Lippmann, R.P., and B. Gold, “Neural-Net Classifier Useful for Speech Recognition,” IEEE 
International Conference -Neural Networks, Vol. 4, San Diego, pp. 417-425, 1987. 

Pal, S.K. and S. Mitra, Neuro-Fuzzy Pattern Recognition Methods in Soft Computing, John Wiley & 
Sons, Inc., New York, 1999. 


9 


